Every machine vision vendor pitch now includes some version of the same diagram: a cloud icon with a GPU in it, an arrow, and a small box at the edge labeled “inference.” It’s a clean picture. The reality of getting a defect-detection model from that cloud icon onto a working line is messier, and the mess is exactly where plant engineers need to spend their attention. The architecture itself — train in the cloud, infer at the edge — is sound and increasingly the right default. The failure mode isn’t the pattern. It’s treating the handoff between data science and the control system as a file copy instead of an engineering interface.
Why this pattern won and rule-based vision didn’t disappear
Classic rule-based machine vision — blob analysis, edge detection, template matching, gauging with tools like those in Cognex VisionPro or Keyence’s vision suites — still wins on jobs with well-defined geometry: presence/absence, dimensional gauging, barcode reads, position verification. It’s deterministic, it’s auditable, and a controls engineer can explain exactly why it failed a part. Deep learning-based vision earns its keep on the harder problem: surface defects with high visual variability — scratches, contamination, weld porosity, cosmetic flaws, texture anomalies — where writing explicit rules for every acceptable variation is impractical.
Training those models is a GPU-heavy, iterative job that belongs in the cloud or in an on-prem data center with real compute: labeling, augmentation, architecture selection, hyperparameter sweeps, validation against holdout sets. Running inference on that trained model at the rate a line demands — often tens to hundreds of milliseconds per frame, synchronized with a PLC scan cycle — is a different problem with different constraints, and it belongs at the edge, physically close to the camera and the actuator that has to reject the part.
Where inference actually runs, and why it matters which box
“Edge” isn’t one place. In practice you’re choosing among a few real options, and the choice changes your latency budget and your integration path:
- Smart camera or vision controller with onboard inference — Cognex, Keyence, and others now ship models that run directly on the camera’s compute module. Lowest latency, tightest coupling to the imaging pipeline, least flexibility for swapping models across camera types.
- Edge PC or industrial gateway sitting between camera and PLC — an NVIDIA Jetson-class device or an industrial PC running a containerized inference runtime (ONNX Runtime, TensorRT), communicating results to the control system over OPC UA, EtherCAT, or a discrete I/O reject signal. More compute headroom, easier to update, one more network hop and one more point of failure to manage.
- Inference embedded in the edge controller itself — Beckhoff’s TwinCAT Vision and its machine-learning extensions, Rockwell’s edge compute modules paired with FactoryTalk, and Siemens’ Industrial Edge platform with its app-based model deployment are all pushing inference into the same runtime that’s doing motion and I/O. This is the tightest integration and the direction the major platforms are clearly headed, because it collapses the “how do I get a reject signal back to the PLC in real time” problem into infrastructure the controls team already owns.
The decision isn’t just about latency. It’s about who owns the box. An inference workload living inside TwinCAT or Industrial Edge is something your controls engineer can version, back up, and reason about using the same tools and access controls as the rest of the automation stack. An inference workload on a separate edge PC bolted next to the panel is IT’s problem, or nobody’s problem, until it needs a security patch nobody scheduled.
Setting a real latency and determinism budget
Rule-based vision tools typically execute in single-digit milliseconds on dedicated hardware, and that number is stable — the same input produces the same timing every time, which is what “deterministic” actually means on a line. Deep learning inference, even optimized with TensorRT or OpenVINO and quantized to INT8, tends to run slower and with more variance, particularly on shared or general-purpose edge hardware. The variance, not the average latency, is the thing that breaks a line. A reject actuator triggered by a PLC scan cycle needs the inference result inside a bounded window, every cycle, not “usually within budget.”
The practical answer is to decouple the two timing domains deliberately. Let the vision model produce a classification or confidence score asynchronously, and let the PLC’s deterministic logic — a tracking encoder position, a physical buffer zone, a delay timer sized to worst-case inference latency plus margin — handle the actual reject actuation. Don’t ask a neural network to be your real-time control loop. Ask it to be an input to one, with a known worst-case delay built into the mechanical or logical design. If your line speed doesn’t allow enough buffer distance for that worst-case delay, you’ve learned something important before commissioning, not after.
Model drift belongs in the quality system, not a notebook
This is the part most pilots get wrong. A data science team trains a model, validates it against a curated test set, deploys it, and considers the job done. Then lighting changes with the season, a new material lot shows subtly different surface texture, a camera gets bumped during a changeover, and the model’s false-reject or false-accept rate drifts — quietly, because nobody built a mechanism to notice.
Drift monitoring has to be wired into the same quality loop that already governs your SPC and non-conformance tracking, not left running as a standalone dashboard a data scientist checks occasionally. Concretely, that means:
- Every inference result — image reference, confidence score, model version, camera/station ID — logged with the same discipline as any other quality measurement, ideally into your MES or a quality data historian, not just an edge device’s local log.
- Confidence-score distributions tracked over time like any other process parameter, with control limits that trigger a review, not just a raw accuracy number computed after the fact.
- A defined escalation path when drift is detected: who reviews it, who approves a retraining cycle, who validates the retrained model against a golden sample set before it goes back to the line. This is a change-control process, and it should look like one, with the same rigor you’d apply to changing a torque spec.
- Model version tied explicitly to the lot and time window it inspected, so a quality escape investigation can answer “which model made this call” as cleanly as it can answer “which operator ran this shift.”
Plants that get this right treat the model version as a traceable quality parameter, on par with gauge calibration status. Plants that get it wrong end up with a defect-detection system nobody fully trusts, because nobody can say with confidence what the model was actually doing on any given day — which defeats the entire point of adding AI vision in the first place.
The decision, plainly
If your defect problem is high-variability surface or cosmetic inspection that rule-based tools genuinely struggle with, cloud training paired with edge inference is the right architecture, and 2026’s tooling from Beckhoff, Rockwell, and Siemens is making that integration meaningfully easier than it was even a couple of years ago. But don’t buy the clean diagram. Budget real engineering time for three things: deciding which physical box does inference and who owns it, designing the control logic so a variable-latency model never sits inside your deterministic reject loop, and building drift monitoring into your existing quality system instead of letting it live in a notebook disconnected from the line. Skip any of the three and you haven’t deployed a vision system — you’ve deployed a demo that happens to be running in production.
This article was written with the assistance of artificial intelligence. While we aim for accuracy, the information may be incomplete, out of date, or incorrect, and should be independently verified before you rely on it for any decision. It is provided for general information only and does not constitute professional advice.
